Add audio translation task type and provider #335
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
julien-nc
commented
Feb 4, 2026
julien-nc
commented
- New audio2audio:translate task type (will be created in server soon)
- Audio translation provider
- Factorize translation logic in a service
- Use correct user language for IL10N text translations happening in the task
julien-nc
changed the title
julien-nc
force-pushed
the
enh/noid/audio-translation
branch
3 times, most recently
from
f3a6a09 to
0c8d7be
Compare
…, factorize translation logic in a service use the correct user language for text translations happening in the task Signed-off-by: Julien Veyssier <julien-nc@posteo.net>
julien-nc
force-pushed
the
enh/noid/audio-translation
branch
from
0c8d7be to
2e45e17
Compare
kyteinsky
reviewed
Feb 4, 2026
Comment on lines
+75
to
+79
| 'text_output' => new ShapeDescriptor( | ||
| $this->l->t('Text output'), | ||
| $this->l->t('The text translation'), | ||
| EShapeType::Text, | ||
| ), |
Contributor
There was a problem hiding this comment.
maybe the transcribed text of the audio input can be sent back too, it's already computed.
Comment on lines
+79
to
+81
| public function getOptionalInputShape(): array { | ||
| return []; | ||
| } |
Contributor
There was a problem hiding this comment.
wdyt of adding the voice and speed optional params here too, same as the text-to-speech task?
integration_openai/lib/TaskProcessing/TextToSpeechProvider.php
Lines 64 to 84 in dfd4567
| $this->logger->warning('Text to speech generation failed: no speech returned'); | ||
| throw new ProcessingException('Text to speech generation failed: no speech returned'); | ||
| } | ||
| $translatedAudio = $includeWatermark ? $this->watermarkingService->markAudio($apiResponse['body']) : $apiResponse['body']; |
Contributor
There was a problem hiding this comment.
maybe better to watermark the transcript of the input audio so the translated text and the translated audio both have the watermark in the target language
Comment on lines
+157
to
+166
| if ($includeWatermark) { | ||
| if ($userId !== null) { | ||
| $user = $this->userManager->getExistingUser($userId); | ||
| $lang = $this->l10nFactory->getUserLanguage($user); | ||
| $l = $this->l10nFactory->get(Application::APP_ID, $lang); | ||
| $ttsPrompt .= "\n\n" . $l->t('This was generated using Artificial Intelligence.'); | ||
| } else { | ||
| $ttsPrompt .= "\n\n" . $this->l->t('This was generated using Artificial Intelligence.'); | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
this can also work but it would add the text/audio in the user's language which may or may not be the target language.
Comment on lines
+109
to
+110
| $context->registerTaskProcessingTaskType(AudioToAudioTranslateTaskType::class); | ||
| $context->registerTaskProcessingProvider(AudioToAudioTranslateProvider::class); |
Contributor
There was a problem hiding this comment.
checks for STT and TTS providers would be nice too
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.